DCR Import to Reltio

OCE Personal (OCE-P) and External Data Change Requests (DCR) to Reltio

Data Change Requests (DCR) that are created in OCE sales and external systems are being processed and posted to Reltio MDM using IDP pipeline.

DCR End to End Data Flow

Templates and best practices

DCR's can be processed into Reltio using two pipelines. Choose the right pipeline based on the requirements.

  • MDM_Load_Reltio_OCE_DCR - Existing pipeline for the DCR's created from one OCEP system (local DCR's) to one Reltio tenant. See config section Import DCR Pipeline Template under configure the parameters.

  • MDM_Load_Reltio_OCE_DCR_GBL - Newly introduced pipeline capable for local plus global use cases where DCR's are created across multiple OCEP systems to one IDP to post to one or multiple Reltio tenants. Refer to the config section Import Global DCR Pipeline Template under  to configure the parameters.

Note:   

Fresh implementation can choose the global pipeline template, which supports both local and global use cases.

Process Flow

  • TSK_OCEP_LOAD: Salesforce to IDP connector exports the OCE data from salesforce application and load into snowflake staging tables.

    Note:   

    This TSK_OCEP_LOAD step needs to be added manually for each OCEP system that is to be integrated within this IDP platform for the global use case.

  • Stage External DCR: LEXI process dumps the external DCR's into IDP S3 location, these files are loaded into staging table using S3Conector plug-in for further processing.

  • Build DCR:

    • Push External DCR - Data loaded into MDM_EXTERNAL_DCR table is pushed into MDM_Canonical schema C_DCR table with DCR details.

    • Populate/Hold OCE DCR: OCE DCR's are logged into an intermediate table called ODP_CORE_LOG.MDM_DCR_PUB_CNTL. This process validates the data, segregates data from multiple OCE tables and puts it into a control table for easy processing. If multiple DCRs are created on the same profile, only one DCR is processed in the first run and the others is processed in the subsequent run. An example: A new HCO (DCR1) and another DCR for affiliation with a new or existing HCP to the new HCO (DCR2). Both DCRs get loaded into the same batch. We can keep DCR2 on hold until DCR1 is processed.

    • Auto Approve/Reject DCR - OCE DCR's that are auto-approved is set with a status of either AUTO_APPROVED or AUTO_REJECTED in the MDM_DCR_PUB_CNTL table based on configurations.

    • Canonical Push - Valid DCR's that are available in MDM_DCR_PUB_CNTL with 'Active' status is transformed to MDM expected JSON format and moved to C_DCR table for Inbound plugin to push. See DCR Flow from canonical to Reltio

  • Load To Reltio: JSON payload created in C_DCR table with PENDING_PLUGIN status are pushed into Reltio tenant based on the Reltio connection configuration.

    Note:   

    Do not repeat this step for different Reltio tenants. Global DCR's is submitted to the right Reltio tenant along with the anypoint message queue details based on the global configuration. For local, the default Reltio connection specified in the pipeline is considered automatically. 

DCR Flow from canonical to Reltio

Overview

Dcr's are received from OCE system into the canonical schema tables. When the Dcr's are received, they do not have attribute level uri's, which are required before submitting the Dcr's to Reltio. This attribute enrichment happens in the canonical schema by making a get entity call to Reltio and fetching the required uri's into the DCR before it can be posted to Reltio. The input table C_DCR is holds the DCR json as received from OCE. Inbound batch processing enritches the Uri's in the JSON and posts it to Reltio. The process flow etc. is explained further in the following section.

Input Table

The input table that hold the incoming DCR has the following important fields.

COLUMN_NAME DATA TYPE PRIMARY KEY COMMENT
DCR_ID VARCHAR Y Auto generated Internal ID 
MDM_URI VARCHAR   Uri of the DCR after it is posted to Reltio
SOURCE_ID VARCHAR   Id of the DCR in the source system
SOURCE_NAME VARCHAR   identifies where the DCR came from 
COUNTRY VARCHAR    
JSON VARCHAR   change requests for the DCR 
LOAD_STATUS VARCHAR   initially PENDING_PLUGIN
LOAD_DETAILS VARCHAR   empty when received

Load Status and Load Details

The load status can have different values depending on the stage of the DCR. It is explained below.

# LOAD_STATUS LOAD_DETAILS ACTION
1 PENDING_PLUGIN EMPTY When the dcr is received from external or oce sales system
2 ERROR

This column can start can state the stage at which this error happened and the details related to this. The various possibilities of the error are explained below

EMPTYDCR_ERROR -- when the payload is not able to detect any changes that need to be posted.

GET_ERROR -- when there is an error or exception while getting the entity from retio tenant for an update DCR.

PAYLOAD_ERROR -- when there is an error while parsing the payload or processing the payload for getting uri's for enriching the changed attributes.

 

PARSECOMMENTS_ERROR – when there was an error parsing the comments which were send with the DCR. 

CREATE_ERROR -- while Posting Creating DCR request Reltio did not return a URI back  indicating that Dcr was not created successfully.

ADDINFO_ERROR -- when there is error/exception adding external Info to DCR. The Error can clear the created in previous step.

STARTDCR_ERROR - while starting DCR we did not get OK status back. The Error can clear the created in previous step.

 

After the dcr is processed once it can have these status's

All records with ERROR status is picked up and send to Lexi system Informing them of the cause of the issue. 
3

ERROR_ACKNOWLEDGE

 Same as before After Lexi has received the ERROR status the status can change to this
4

SUCCESSFUL

 The DCR was created successfully and load details can have the uri of the DCR

No Action

 

 
  PENDING_PLUGIN Details of the problem. As to why DCR could not be posted and it is attempted again This can happen when the DCR Reltio throws an error like System busy or timeout. The DCR can remain in PENGING_PLUGIN state and is attempted for processing during next run.

Distributed Tracing

After the DCR is processed, a message is sent to the distributed tracing queue with the result of the DCR. The message can have a stage and status embedded in it, indicating the state of the DCR after the processing is completed. Stage and status can have the following values:

Status Stage Description
PENDING SUCCESSFUL Dcr was processed successfully and is in pending status
ERROR PENDING The dcr is in error but it is recoverable and is attempted again
ERROR ERROR Dcr is in permanent error state and Lexi is informed